Celestin Apprentice 7

home *** CD-ROM | disk | FTP | other *** search

/ Celestin Apprentice 7 / Apprentice-Release7.iso / Environments / PowerLisp 2.01 / Supplemental Documentation / Documentation / Chapter 23. File System Interf… < prev next >

Wrap

Text File | 1995-03-27 | 127.5 KB | 2,757 lines | [TEXT/ROSA]

Common Lisp the Language, 2nd Edition ------------------------------------------------------------------------------- 23. File System Interface A frequent use of streams is to communicate with a file system to which groups of data (files) can be written and from which files can be retrieved. Common Lisp defines a standard interface for dealing with such a file system. This interface is designed to be simple and general enough to accommodate the facilities provided by ``typical'' operating system environments within which Common Lisp is likely to be implemented. The goal is to make Common Lisp programs that perform only simple operations on files reasonably portable. To this end, Common Lisp assumes that files are named, that given a name one can construct a stream connected to a file of that name, and that the names can be fit into a certain canonical, implementation-independent form called a pathname. Facilities are provided for manipulating pathnames, for creating streams connected to files, and for manipulating the file system through pathnames and streams. ------------------------------------------------------------------------------- * File Names o Pathnames o Case Conventions o Structured Directories o Extended Wildcards o Logical Pathnames + Syntax of Logical Pathname Namestrings + Parsing of Logical Pathname Namestrings + Using Logical Pathnames + Examples of the Use of Logical Pathnames + Discussion of Logical Pathnames o Pathname Functions * Opening and Closing Files * Renaming, Deleting, and Other File Operations * Loading Files * Accessing Directories ------------------------------------------------------------------------------- 23.1. File Names Common Lisp programs need to use names to designate files. The main difficulty in dealing with names of files is that different file systems have different naming formats for files. For example, here is a table of several file systems (actually, operating systems that provide file systems) and what equivalent file names might look like for each one: System File Name ================================================== TOPS-20 <LISPIO>FORMAT.FASL.13 TOPS-10 FORMAT.FAS[1,4] ITS LISPIO;FORMAT FASL MULTICS >udd>LispIO>format.fasl TENEX <LISPIO>FORMAT.FASL;13 VAX/VMS [LISPIO]FORMAT.FAS;13 UNIX /usr/lispio/format.fasl ================================================== It would be impossible for each program that deals with file names to know about each different file name format that exists; a new Common Lisp implementation might use a format different from any of its predecessors. Therefore, Common Lisp provides two ways to represent file names: namestrings, which are strings in the implementation-dependent form customary for the file system, and pathnames, which are special abstract data objects that represent file names in an implementation-independent way. Functions are provided to convert between these two representations, and all manipulations of files can be expressed in machine-independent terms by using pathnames. In order to allow Common Lisp programs to operate in a network environment that may have more than one kind of file system, the pathname facility allows a file name to specify which file system is to be used. In this context, each file system is called a host, in keeping with the usual networking terminology. [change_begin] Different hosts may use different notations for file names. Common Lisp allows customary notation to be used for each host, but also supports a system of logical pathnames that provides a standard framework for naming files in a portable manner (see section 23.1.5). [change_end] ------------------------------------------------------------------------------- * Pathnames * Case Conventions * Structured Directories * Extended Wildcards * Logical Pathnames o Syntax of Logical Pathname Namestrings o Parsing of Logical Pathname Namestrings o Using Logical Pathnames o Examples of the Use of Logical Pathnames o Discussion of Logical Pathnames * Pathname Functions ------------------------------------------------------------------------------- 23.1.1. Pathnames All file systems dealt with by Common Lisp are forced into a common framework, in which files are named by a Lisp data object of type pathname. A pathname always has six components, described below. These components are the common interface that allows programs to work the same way with different file systems; the mapping of the pathname components into the concepts peculiar to each file system is taken care of by the Common Lisp implementation. host The name of the file system on which the file resides. device Corresponds to the ``device'' or ``file structure'' concept in many host file systems: the name of a (logical or physical) device containing files. directory Corresponds to the ``directory'' concept in many host file systems: the name of a group of related files (typically those belonging to a single user or project). name The name of a group of files that can be thought of as the ``same'' file. type Corresponds to the ``filetype'' or ``extension'' concept in many host file systems; identifies the type of file. Files with the same names but different types are usually related in some specific way, for instance, one being a source file, another the compiled form of that source, and a third the listing of error messages from the compiler. version Corresponds to the ``version number'' concept in many host file systems. Typically this is a number that is incremented every time the file is modified. Note that a pathname is not necessarily the name of a specific file. Rather, it is a specification (possibly only a partial specification) of how to access a file. A pathname need not correspond to any file that actually exists, and more than one pathname can refer to the same file. For example, the pathname with a version of ``newest'' may refer to the same file as a pathname with the same components except a certain number as the version. Indeed, a pathname with version ``newest'' may refer to different files as time passes, because the meaning of such a pathname depends on the state of the file system. In file systems with such facilities as ``links,'' multiple file names, logical devices, and so on, two pathnames that look quite different may turn out to address the same file. To access a file given a pathname, one must do a file system operation such as open. Two important operations involving pathnames are parsing and merging. Parsing is the conversion of a namestring (which might be something supplied interactively by the user when asked to supply the name of a file) into a pathname object. This operation is implementation-dependent, because the format of namestrings is implementation-dependent. Merging takes a pathname with missing components and supplies values for those components from a source of defaults. Not all of the components of a pathname need to be specified. If a component of a pathname is missing, its value is nil. Before the file system interface can do anything interesting with a file, such as opening the file, all the missing components of a pathname must be filled in (typically from a set of defaults). Pathnames with missing components may be used internally for various purposes; in particular, parsing a namestring that does not specify certain components will result in a pathname with missing components. [change_begin] X3J13 voted in January 1989 (PATHNAME-UNSPECIFIC-COMPONENT) to permit any component of a pathname to have the value :unspecific, meaning that the component simply does not exist, for file systems in which such a value makes sense. (For example, a UNIX file system usually does not support version numbers, so the version component of a pathname for a UNIX host might be :unspecific. Similarly, the file type is usually regarded in a UNIX file system as the part of a name after a period, but some file names contain no periods and therefore have no file types.) When a pathname is converted to a namestring, the values nil and :unspecific have the same effect: they are treated as if the component were empty (that is, they each cause the component not to appear in the namestring). When merging, however, only a nil value for a component will be replaced with the default for that component; the value :unspecific will be left alone as if the field were filled. The results are undefined if :unspecific is supplied to a file system in a component for which :unspecific does not make sense for that file system. Programming hint: portable programs should be prepared to handle the value :unspecific in the device, directory, type, or version field in some implementations. Portable programs should not explicitly place :unspecific in any field because it might not be permitted in some situations, but portable programs may sometimes do so implicitly (by copying such a value from another pathname, for example). [change_end] [old_change_begin] A component of a pathname can also be the keyword :wild. This is only useful when the pathname is being used with a directory-manipulating operation, where it means that the pathname component matches anything. The printed representation of a pathname typically designates :wild by an asterisk; however, this is host-dependent. [old_change_end] See section 23.1.4 for a discussion of new wildcard pathname facilities. What values are allowed for components of a pathname depends, in general, on the pathname's host. However, in order for pathnames to be usable in a system-independent way, certain global conventions are adhered to. These conventions are stronger for the type and version than for the other components, since the type and version are explicitly manipulated by many programs, while the other components are usually treated as something supplied by the user that just needs to be remembered and copied from place to place. The type is always a string or nil or :wild. It is expected that most programs that deal with files will supply a default type for each file. The version is either a positive integer or a special symbol. The meanings of nil and :wild have been explained above. The keyword :newest refers to the largest version number that already exists in the file system when reading a file, or to a version number greater than any already existing in the file system when writing a new file. Some Common Lisp implementors may choose to define other special version symbols. Some semi-standard names, suggested but not required to be supported by every Common Lisp implementation, are :oldest, to refer to the smallest version number that exists in the file system; :previous, to refer to the version previous to the newest version; and :installed, to refer to a version that is officially installed for users (as opposed to a working or development version). Some Common Lisp implementors may also choose to attach a meaning to non-positive version numbers (a typical convention is that 0 is synonymous with :newest and -1 with :previous), but such interpretations are implementation-dependent. The host may be a string, indicating a file system, or a list of strings, of which the first names the file system and the rest may be used for such a purpose as inter-network routing. [old_change_begin] The device, directory, and name can each be a string (with host-dependent rules on allowed characters and length) or possibly some other Common Lisp data structure (in which case such a component is said to be structured and has an implementation-dependent format). Structured components may be used to handle such file system features as hierarchical directories. Common Lisp programs do not need to know about structured components unless they do host-dependent operations. Specifying a string as a pathname component for a host that requires a structured component will cause conversion of the string to the appropriate form. [old_change_end] [change_begin] X3J13 voted in June 1989 (PATHNAME-SUBDIRECTORY-LIST) to define a specific format for structured directories (see section 23.1.3). X3J13 voted in June 1989 (PATHNAME-COMPONENT-VALUE) to approve the following clarifications and specifications of precisely what are valid values for the various components of a pathname. Pathname component value strings never contain the punctuation characters that are used to separate fields in a namestring (for example, slashes and periods as used in UNIX file systems). Punctuation characters appear only in namestrings. Characters used as punctuation can appear in pathname component values with a non-punctuation meaning if the file system allows it (for example, UNIX file systems allow a file name to begin with a period). When examining pathname components, conforming programs must be prepared to encounter any of the following siutations: * Any component can be nil, which means the component has not been specified. * Any component can be :unspecific, which means the component has no meaning in this particular pathname. * The device, directory, name, and type can be strings. * The host can be any object, at the discretion of the implementation. * The directory can be a list of strings and symbols as described in section 23.1.3. * The version can be any symbol or any integer. The symbol :newest refers to the largest version number that already exists in the file system when reading, overwriting, appending, superseding, or directory-listing an existing file; it refers to the smallest version number greater than any existing version number when creating a new file. Other symbols and integers have implementation-defined meaning. It is suggested, but not required, that implementations use positive integers starting at 1 as version numbers, recognize the symbol :oldest to designate the smallest existing version number, and use keyword symbols for other special versions. When examining wildcard components of a wildcard pathname, conforming programs must be prepared to encounter any of the following additional values in any component or any element of a list that is the directory component: * The symbol :wild, which matches anything. * A string containing implementation-dependent special wildcard characters. * Any object, representing an implementation-dependent wildcard pattern. When constructing a pathname from components, conforming programs must follow these rules: * Any component may be nil. Specifying nil for the host may, in some implementations, result in using a default host rather than an actual nil value. * The host, device, directory, name, and type may be strings. There are implementation-dependent limits on the number and type of characters in these strings. A plausible assumption is that letters (of a single case) and digits are acceptable to most file systems. section 23.1.3. There are implementation-dependent limits on the length and contents of the list. * The version may be :newest. * Any component may be taken from the corresponding component of another pathname. When the two pathnames are for different file systems (in implementations that support multiple file systems), an appropriate translation occurs. If no meaningful translation is possible, an error is signaled. The definitions of ``appropriate'' and ``meaningful'' are implementation-dependent. * When constructing a wildcard pathname, the name, type, or version may be :wild, which matches anything. * An implementation might support other values for some components, but a portable program should not use those values. A conforming program can use implementation-dependent values but this can make it non-portable; for example, it might work only with UNIX file systems. [change_end] The best way to compare two pathnames for equality is with equal, not eql. (On pathnames, eql is simply the same as eq.) Two pathname objects are equal if and only if all the corresponding components (host, device, and so on) are equivalent. (Whether or not uppercase and lowercase letters are considered equivalent in strings appearing in components depends on the file name conventions of the file system.) Pathnames that are equal should be functionally equivalent. [old_change_begin] Some host file systems have features that do not fit into this pathname model. For instance, directories might be accessible as files; there might be complicated structure in the directories or names; or there might be a way to specify a directory relative to a ``current'' directory, such as the < syntax in MULTICS or the special ``..'' file name of UNIX. Such features are not allowed for by the standard Common Lisp file system interface. An implementation is free to accommodate such features in its pathname representation and provide a parser that can process such specifications in namestrings; such features are then likely to work within that single implementation. However, note that once a program depends explicitly on any such features, it will not be portable. [old_change_end] [change_begin] X3J13 voted in June 1989 (PATHNAME-SUBDIRECTORY-LIST) to define a specific format for structured directories (see section 23.1.3), so some of the specific examples in the previous paragraph no longer apply, but the principle is still correct. [change_end] ------------------------------------------------------------------------------- 23.1.2. Case Conventions [change_begin] Issues of alphabetic case in pathnames are a major source of problems. In some file systems, the customary case is lowercase, in some uppercase, in some mixed. Some file systems are case-sensitive (that is, they treat FOO and foo as different file names) and others are not. There are two kinds of pathname case portability problems: moving programs from one Common Lisp to another, and moving pathname component values from one file system to another. The solution to the first problem is the requirement that all Common Lisp implementations that support a particular file system must use compatible representations for pathname component values. The solution to the second problem is the use of a common representation for the least-common-denominator pathname component values that exist on all interesting file systems. Requiring a common representation directly conflicts with the desire among programmers that use only one file system to work with the local conventions and to ignore issues of porting to other file systems. The common representation cannot be the same as local (varying) conventions. X3J13 voted in June 1989 (PATHNAME-COMPONENT-CASE) to add a keyword argument :case to each of the functions make-pathname, pathname-host, pathname-device, pathname-directory, pathname-name, and pathname-type. The possible values for the argument are :common and :local. The default is :local. The value :local means that strings given to make-pathname or returned by any of the pathname component accessors follow the local file system's conventions for alphabetic case. Strings given to make-pathname will be used exactly as written if the file system supports both cases. If the file system supports only one case, the strings will be translated to that case. The value :common means that strings given to make-pathname or returned by any of the pathname component accessors follow this common convention: * All uppercase means that a file system's customary case will be used. * All lowercase means that the opposite of the customary case will be used. * Mixed case represents itself. Uppercase is used as the common case for no better reason than consistency with Lisp symbols. The second and third points allow translation from local representation to common and back to be information-preserving. (Note that translation from common to local representation and back may or may not be information-preserving, depending on the nature of the local representation.) Namestrings always use :local file system case conventions. Finally, merge-pathnames and translate-pathname map customary case in the input pathnames into customary case in the output pathname. Examples of possible use of this convention: * TOPS-20 is case-sensitive and prefers uppercase, translating lowercase to uppercase unless escaped with ^V; for a TOPS-20-based file system, a Common Lisp implementation should use identical representations for common and local. * UNIX is case-sensitive and prefers lowercase; for a UNIX-based file system, a Common Lisp implementation should translate between common and local representations by inverting the case of non-mixed-case strings. * VAX/VMS is uppercase-only (that is, the file system translates all file name arguments to uppercase); for a VAX/VMS-based file system, a Common Lisp implementation should translate common representation to local by converting to uppercase and should translate local representation to common with no change. * The Macintosh operating system is case-insensitive and prefers lowercase, but remembers the cases of letters actually used to name a file; for a Macintosh-based file system, a Common Lisp implementation should translate between common and local representations by inverting the case of non-mixed-case strings and should ignore case when determining whether two pathnames are equal. Here are some examples of this behavior. Assume that the host T runs TOPS-20, U runs UNIX, V runs VAX/VMS, and M runs the Macintosh operating system. ;;; Returns two values: the PATHNAME-NAME from a namestring ;;; in :COMMON and :LOCAL representations (in that order). (defun pathname-example (name) (let ((path (parse-namestring name)))) (values (pathname-name path :case :common) (pathname-name path :case :local)))) => "FOO" and Ø"FOO" ;Common Local (pathname-example "T:<ME>FOO.LISP") => "FOO" and "FOO" (pathname-example "T:<ME>foo.LISP") => "FOO" and "FOO" (pathname-example "T:<ME>^Vf^Vo^Vo.LISP") => "foo" and "foo" (pathname-example "T:<ME>TeX.LISP") => "TEX" and "TEX" (pathname-example "T:<ME>T^VeX.LISP") => "TeX" and "TeX" (pathname-example "U:/me/FOO.lisp") => "foo" and "FOO" (pathname-example "U:/me/foo.lisp") => "FOO" and "foo" (pathname-example "U:/me/TeX.lisp") => "TeX" and "TeX" (pathname-example "V:[me]FOO.LISP") => "FOO" and "FOO" (pathname-example "V:[me]foo.LISP") => "FOO" and "FOO" (pathname-example "V:[me]TeX.LISP") => "TEX" and "TEX" (pathname-example "M:FOO.LISP") => "foo" and "FOO" (pathname-example "M:foo.LISP") => "FOO" and "foo" (pathname-example "M:TeX.LISP") => "TeX" and "TeX" The following example illustrates the creation of new pathnames. The name is converted from common representation to local because namestrings always use local conventions. (defun make-pathname-example (h n) (namestring (make-pathname :host h :name n :case :common)) (make-pathname-example "T" "FOO") => "T:FOO" (make-pathname-example "T" "foo") => "T:^Vf^Vo^Vo" (make-pathname-example "T" "TeX") => "T:T^VeX" (make-pathname-example "U" "FOO") => "U:foo" (make-pathname-example "U" "foo") => "U:FOO" (make-pathname-example "U" "TeX") => "U:TeX" (make-pathname-example "V" "FOO") => "V:FOO" (make-pathname-example "V" "foo") => "V:FOO" (make-pathname-example "V" "TeX") => "V:TeX" (make-pathname-example "M" "FOO") => "M:foo" (make-pathname-example "M" "foo") => "M:FOO" (make-pathname-example "M" "TeX") => "M:TeX" A big advantage of this set of conventions is that one can, for example, call make-pathname with :type "LISP" and :case :common, and the result will appear in a namestring as .LISP or .lisp, whichever is appropriate. [change_end] ------------------------------------------------------------------------------- 23.1.3. Structured Directories [change_begin] X3J13 voted in June 1989 (PATHNAME-SUBDIRECTORY-LIST) to define a specific pathname component format for structured directories. The value of a pathname's directory component may be a list. The car of the list should be a keyword, either :absolute or :relative. Each remaining element of the list should be a string or a symbol (see below). Each string names a single level of directory structure and should consist of only the directory name without any punctuation characters. A list whose car is the symbol :absolute represents a directory path starting from the root directory. For example, the list (:absolute) represents the root directory itself; the list (:absolute "foo" "bar" "baz") represents the directory that in a UNIX file system would be called /foo/bar/baz. A list whose car is the symbol :relative represents a directory path starting from a default directory. The list (:relative) has the same meaning as nil and hence normally is not used. The list (:relative "foo" "bar") represents the directory named bar in the directory named foo in the default directory. In place of a string, at any point in the list, a symbol may occur to indicate a special file notation. The following symbols have standard meanings. :wild Wildcard match of one level of directory structure :wild-inferiors Wildcard match of any number of directory levels :up Go upward in directory structure (semantic) :back Go upward in directory structure (syntactic) (See section 23.1.4 for a discussion of wildcard pathnames.) Implementations are permitted to add additional objects of any non-string type if necessary to represent features of their file systems that cannot be represented with the standard strings and symbols. Supplying any non-string, including any of the symbols listed below, to a file system for which it does not make sense signals an error of type file-error. For example, most implementations of the UNIX file system do not support :wild-inferiors. Any directory list in which :absolute or :wild-inferiors is immediately followed by :up or :back is illegal and when processed causes an error to be signaled. The keyword :back has a ``syntactic'' meaning that depends only on the pathname and not on the contents of the file system. The keyword :up has a ``semantic'' meaning that depends on the contents of the file system; to resolve a pathname containing :up to a pathname whose directory component contains only :absolute and strings requires a search of the file system. Note that use of :up instead of :back can result in designating a different actual directory only in file systems that support multiple names for directories, perhaps via symbolic links. For example, suppose that there is a directory link such that (:absolute "X" "Y") is linked to (:absolute "A" "B") and there also exist directories (:absolute "A" "Q") and (:absolute "X" "Q") Then (:absolute "X" "Y" :up "Q") designates (:absolute "A" "Q") but (:absolute "X" "Y" :back "Q") designates (:absolute "X" "Q") If a string is used as the value of the :directory argument to make-pathname, it should be the name of a top-level directory and should not contain any punctuation characters. Specifying a string s is equivalent to specifying the list (:absolute s). Specifying the symbol :wild is equivalent to specifying the list (:absolute :wild-inferiors) (or (:absolute :wild) in a file system that does not support :wild-inferiors). The function pathname-directory always returns nil, :unspecific, or a list-never a string, never :wild. If a list is returned, it is not guaranteed to be freshly consed; the consequences of modifying this list are undefined. In non-hierarchical file systems, the only valid list values for the directory component of a pathname are (:absolute s) (where s is a string) and (:absolute :wild). The keywords :relative, :wild-inferiors, :up, and :back are not used in non-hierarchical file systems. Pathname merging treats a relative directory specially. Let pathname and defaults be the first two arguments to merge-pathnames. If (pathname-directory pathname) is a list whose car is :relative, and (pathname-directory defaults) is a list, then the merged directory is the value of (append (pathname-directory defaults) (cdr ;Remove :relative from the front (pathname-directory pathname))) except that if the resulting list contains a string or :wild immediately followed by :back, both of them are removed. This removal of redundant occurrences of :back is repeated as many times as possible. If (pathname-directory defaults) is not a list or (pathname-directory pathname) is not a list whose car is :relative, the merged directory is the value of (or (pathname-directory pathname) (pathname-directory defaults)) A relative directory in the pathname argument to a function such as open is merged with the value of *default-pathname-defaults* before the file system is accessed. Here are some examples of the use of structured directories. Suppose that host L supports a Symbolics Lisp Machine file system, host U supports a UNIX file system, and host V supports a VAX/VMS file system. (pathname-directory (parse-namestring "V:[FOO.BAR]BAZ.LSP")) => (:ABSOLUTE "FOO" "BAR") (pathname-directory (parse-namestring "U:/foo/bar/baz.lisp")) => (:ABSOLUTE "foo" "bar") (pathname-directory (parse-namestring "U:../baz.lisp")) => (:RELATIVE :UP) (pathname-directory (parse-namestring "U:/foo/bar/../mum/baz")) => (:ABSOLUTE "foo" "bar" :UP "mum") (pathname-directory (parse-namestring "U:bar/../../ztesch/zip")) => (:RELATIVE "bar" :UP :UP "ztesch") (pathname-directory (parse-namestring "L:>foo>**>bar>baz.lisp")) => (:ABSOLUTE "FOO" :WILD-INFERIORS "BAR") (pathname-directory (parse-namestring "L:>foo>*>bar>baz.lisp")) => (:ABSOLUTE "FOO" :WILD "BAR") [change_end] ------------------------------------------------------------------------------- 23.1.4. Extended Wildcards [change_begin] Some file systems provide more complex conventions for wildcards than simple component-wise wildcards representable by :wild. For example, the namestring "F*O" might mean a normal three-character name; a three-character name with the middle character wild; a name with at least two characters, beginning with F and ending with O; or perhaps a wild match spanning multiple directories. Similarly, the namestring ">foo>**>bar>" might imply that the middle directory is named "**"; the middle directory is :wild; there are zero or more middle directories that are :wild; or perhaps that the middle directory name matches any two-letter name. Some file systems support even more complex wildcards, such as regular expressions. X3J13 voted in June 1989 (PATHNAME-WILD) to provide some facilities for dealing with more general wildcard pathnames in a fairly portable manner. [Function] wild-pathname-p pathname &optional field-key Tests a pathname for the presence of wildcard components. If the first argument is not a pathname, string, or file stream, an error of type type-error is signaled. If no field-key is provided, or the field-key is nil, the result is true if and only if pathname has any wildcard components. If a non-null field-key is provided, it must be one of :host, :device, :directory, :name, :type, or :version. In this case, the result is true if and only if the indicated component of pathname is a wildcard. Note that X3J13 voted in June 1989 (PATHNAME-COMPONENT-VALUE) to specify that an implementation need not support wildcards in all fields; the only requirement is that the name, type, or version may be :wild. However, portable programs should be prepared to encounter either :wild or implementation-dependent wildcards in any pathname component. The function wild-pathname-p provides a portable way for testing the presence of wildcards. [Function] pathname-match-p pathname wildname This predicate is true if and only if the pathname matches the wildname. The matching rules are implementation-defined but should be consistent with the behavior of the directory function. Missing components of wildname default to :wild. If either argument is not a pathname, string, or file stream, an error of type type-error is signaled. It is valid for pathname to be a wild pathname; a wildcard field in pathname will match only a wildcard field in wildname; that is, pathname-match-p is not commutative. It is valid for wildname to be a non-wild pathname; I believe that in this case pathname-match-p will have the same behavior as equal, though the X3J13 specification did not say so. [Function] translate-pathname source from-wildname to-wildname &key Translates the pathname source, which must match from-wildname, into a corresponding pathname (call it result), which is constructed so as to match to-wildname, and returns result. The pathname result is a copy of to-wildname with each missing or wildcard field replaced by a portion of source; for this purpose a wildcard field is a pathname component with a value of :wild, a :wild element of a list-valued directory component, or an implementation-defined portion of a component, such as the * in the complex wildcard string "foo*bar" that some implementations support. An implementation that adds other wildcard features, such as regular expressions, must define how translate-pathname extends to those features. A missing field is a pathname component that is nil. The portion of source that is copied into result is implementation-defined. Typically it is determined by the user interface conventions of the file systems involved. Usually it is the portion of source that matches a wildcard field of from-wildname that is in the same position as the missing or wildcard field of to-wildname. If there is no wildcard field in from-wildname at that position, then usually it is the entire corresponding pathname component of source or, in the case of a list-valued directory component, the entire corresponding list element. For example, if the name components of source, from-wildname, and to-wildname are "gazonk", "gaz*", and "h*" respectively, then in most file systems the wildcard fields of the name component of from-wildname and to-wildname are each "*", the matching portion of source is "onk", and the name component of result is "honk"; however, the exact behavior of translate-pathname is not dictated by the Common Lisp language and may vary according to the user interface conventions of the file systems involved. During the copying of a portion of source into result, additional implementation-defined translations of alphabetic case or file naming conventions may occur, especially when from-wildname and to-wildname are for different hosts. If any of the first three arguments is not a pathname, string, or file stream, an error of type type-error is signaled. It is valid for source to be a wild pathname; in general this will produce a wild result pathname. It is valid for from-wildname or to-wildname or both to be non-wild. An error is signaled if the source pathname does not match the from-wildname, that is, if (pathname-match-p source from-wildname) would not be true. There are no specified keyword arguments for translate-pathname, but implementations are permitted to extend it by adding keyword arguments. There is one specified return value from translate-pathname; implementations are permitted to extend it by returning additional values. Here is an implementation suggestion. One file system performs this operation by examining corresponding pieces of the three pathnames in turn, where a piece is a pathname component or a list element of a structured component such as a hierarchical directory. Hierarchical directory elements in from-wildname and to-wildname are matched by whether they are wildcards, not by depth in the directory hierarchy. If the piece in to-wildname is present and not wild, it is copied into the result. If the piece in to-wildname is :wild or nil, the corresponding piece in source is copied into the result. Otherwise, the piece in to-wildname might be a complex wildcard such as "foo*bar"; the portion of the piece in source that matches the wildcard portion of the corresponding piece in from-wildname (or the entire source piece, if the from-wildname piece is not wild and therefore equals the source piece) replaces the wildcard portion of the piece in to-wildname and the value produced is used in the result. X3J13 voted in June 1989 (PATHNAME-COMPONENT-CASE) to require translate-pathname to map customary case in argument pathnames to the customary case in returned pathnames (see section 23.1.2). Here are some examples of the use of the new wildcard pathname facilities. These examples are not portable. They are written to run with particular file systems and particular wildcard conventions and are intended to be illustrative, not prescriptive. Other implementations may behave differently. (wild-pathname-p (make-pathname :name :wild)) => t (wild-pathname-p (make-pathname :name :wild) :name) => t (wild-pathname-p (make-pathname :name :wild) :type) => nil (wild-pathname-p (pathname "S:>foo>**>")) => t ;Maybe (wild-pathname-p (make-pathname :name "F*O")) => t ;Probably One cannot rely on rename-file to handle wild pathnames in a predictable manner. However, one can use translate-pathname explicitly to control the process. (defun rename-files (from to) "Rename all files that match the first argument by translating their names to the form of the second argument. Both arguments may be wild pathnames." (dolist (file (directory from)) ;; DIRECTORY produces only pathnames that match from-wildname. (rename-file file (translate-pathname file from to)))) Assuming one particular set of popular wildcard conventions, this function might exhibit the following behavior. Not all file systems will run this example exactly as written. (rename-files "/usr/me/*.lisp" "/dev/her/*.l") renames /usr/me/init.lisp to /dev/her/init.l (rename-files "/usr/me/pcl*/*" "/sys/pcl/*/") renames /usr/me/pcl-5-may/low.lisp to /sys/pcl/pcl-5-may/low.lisp (in some file systems the result might be /sys/pcl/5-may/low.lisp) (rename-files "/usr/me/pcl*/*" "/sys/library/*/") renames /usr/me/pcl-5-may/low.lisp to /sys/library/pcl-5-may/low.lisp (in some file systems the result might be /sys/library/5-may/low.lisp) (rename-files "/usr/me/foo.bar" "/usr/me2/") renames /usr/me/foo.bar to /usr/me2/foo.bar (rename-files "/usr/joe/*-recipes.text" "/usr/jim/personal/cookbook/joe's-*-rec.text") renames /usr/joe/lamb-recipes.text to /usr/jim/personal/cookbook/joe's-lamb-rec.text renames /usr/joe/veg-recipes.text to /usr/jim/personal/cookbook/joe's-veg-rec.text renames /usr/joe/cajun-recipes.text to /usr/jim/personal/cookbook/joe's-cajun-rec.text renames /usr/joe/szechuan-recipes.text to /usr/jim/personal/cookbook/joe's-szechuan-rec.text The following examples use UNIX syntax and the wildcard conventions of one particular version of UNIX. (namestring (translate-pathname "/usr/dmr/hacks/frob.l" "/usr/d*/hacks/*.l" "/usr/d*/backup/hacks/backup-*.*")) => "/usr/dmr/backup/hacks/backup-frob.l" (namestring (translate-pathname "/usr/dmr/hacks/frob.l" "/usr/d*/hacks/fr*.l" "/usr/d*/backup/hacks/backup-*.*")) => "/usr/dmr/backup/hacks/backup-ob.l" The following examples are similar to the preceding examples but use two different hosts; host U supports a UNIX file system and host V supports a VAX/VMS file system. Note the translation of file type (from l to LSP) and the change of alphabetic case conventions. (namestring (translate-pathname "U:/usr/dmr/hacks/frob.l" "U:/usr/d*/hacks/*.l" "V:SYS$DISK:[D*.BACKUP.HACKS]BACKUP-*.*")) => "V:SYS$DISK:[DMR.BACKUP.HACKS]BACKUP-FROB.LSP" (namestring (translate-pathname "U:/usr/dmr/hacks/frob.l" "U:/usr/d*/hacks/fr*.l" "V:SYS$DISK:[D*.BACKUP.HACKS]BACKUP-*.*")) => "V:SYS$DISK:[DMR.BACKUP.HACKS]BACKUP-OB.LSP" The next example is a version of the function translate-logical-pathname (simplified a bit) for a logical host named FOO. The points of interest are the use of pathname-match-p as a :test argument for assoc and the use of translate-pathname as a substrate for translate-logical-pathname. (define-condition logical-translation-error (file-error)) (defun my-translate-logical-pathname (pathname &key rules) (let ((rule (assoc pathname rules :test #'pathname-match-p))) (unless rule (error 'logical-translation-error :pathname pathname)) (translate-pathname pathname (first rule) (second rule)))) (my-translate-logical-pathname "FOO:CODE;BASIC.LISP" :rules '(("FOO:DOCUMENTATION;" "U:/doc/foo/") ("FOO:CODE;" "U:/lib/foo/") ("FOO:PATCHES;*;" "U:/lib/foo/patch/*/"))) => #P"U:/lib/foo/basic.l" [change_end] ------------------------------------------------------------------------------- 23.1.5. Logical Pathnames [change_begin] Pathname values are not portable, but sometimes they must be mentioned in a program (for example, the names of files containing the program and the data used by the program). X3J13 voted in June 1989 (PATHNAME-LOGICAL) to provide some facilities for portable pathname values. The idea is to provide a portable framework for pathname values; these logical pathnames are then mapped to physical (that is, actual) pathnames by a set of implementation-dependent or site-dependent rules. The logical pathname facility therefore separates the concerns of program writing and user software architecture from the details of how a software system is embedded in a particular file system or operating environment. Pathname values are not portable because not all Common Lisp implementations use the same operating system and file name syntax varies widely among operating systems. In addition, corresponding files at two different sites may have different names even when the operating system is the same; for example, they may be on different directories or different devices. The Common Lisp logical pathname system defines a particular pathname structure and namestring syntax that must be supported by all implementations. [Class] logical-pathname This is a subclass of pathname. [change_end] ------------------------------------------------------------------------------- * Syntax of Logical Pathname Namestrings * Parsing of Logical Pathname Namestrings * Using Logical Pathnames * Examples of the Use of Logical Pathnames * Discussion of Logical Pathnames ------------------------------------------------------------------------------- 23.1.5.1. Syntax of Logical Pathname Namestrings [change_begin] The syntax of a logical pathname namestring is as follows: logical-namestring ::= [host :] [;] {directory ;}* [name] [. type [. version] Note that a logical namestring has no device portion. host ::= word directory ::= word | wildcard-word | wildcard-inferiors name ::= word | wildcard-word type ::= word | wildcard-word version ::= word | wildcard-word word ::= {letter digit -}+ wildcard-word ::= [word] * {word *}* [word] wildcard-inferiors ::= ** A word consists of one or more uppercase letters, digits, and hyphens. A wildcard word consists of one or more asterisks, uppercase letters, digits, and hyphens, including at least one asterisk, with no two asterisks adjacent. Each asterisk matches a sequence of zero or more characters. The wildcard word * parses as :wild; all others parse as strings. Lowercase letters may also appear in a word or wildcard word occurring in a namestring. Such letters are converted to uppercase when the namestring is converted to a pathname. The consequences of using other characters are unspecified. The host is a word that has been defined as a logical pathname host by using setf with the function logical-pathname-translations. There is no device, so the device component of a logical pathname is always :unspecific. No other component of a logical pathname can be :unspecific. Each directory is a word, a wildcard word, or ** (which is parsed as :wild-inferiors). If a semicolon precedes the directories, the directory component is relative; otherwise it is absolute. The name is a word or a wildcard word. The type is a word or a wildcard word. The version is a positive decimal integer or the word NEWEST (which is parsed as :newest) or * (which is parsed as :wild). The letters in NEWEST can be in either alphabetic case. The consequences of using any value not specified here as a logical pathname component are unspecified. The null string "" is not a valid value for any component of a logical pathname, since the null string is not a word or a wildcard word. [change_end] ------------------------------------------------------------------------------- 23.1.5.2. Parsing of Logical Pathname Namestrings [change_begin] Logical pathname namestrings are recognized by the functions logical-pathname and translate-logical-pathname. The host portion of the logical pathname namestring and its following colon must appear in the namestring arguments to these functions. The function parse-namestring recognizes a logical pathname namestring when the host argument is logical or the defaults argument is a logical pathname. In this case the host portion of the logical pathname namestring and its following colon are optional. If the host portion of the namestring and the host argument are both present and do not match, an error is signaled. The host argument is logical if it is supplied and came from pathname-host of a logical pathname. Whether a host argument is logical if it is a string equal to a logical pathname host name is implementation-defined. The function merge-pathnames recognizes a logical pathname namestring when the defaults argument is a logical pathname. In this case the host portion of the logical pathname namestring and its following colon are optional. Whether the other functions that coerce strings to pathnames recognize logical pathname namestrings is implementation-defined. These functions include parse-namestring in circumstances other than those described above, merge-pathnames in circumstances other than those described above, the :defaults argument to make-pathname, and the following functions: compile-file file-write-date pathname-name compile-file-pathname host-namestring pathname-type delete-file load pathname-version directory namestring probe-file directory-namestring open rename-file dribble pathname translate-pathname ed pathname-device truename enough-namestring pathname-directory wild-pathname-p file-author pathname-host with-open-file file-namestring pathname-match-p Note that many of these functions must accept logical pathnames even though they do not accept logical pathname namestrings. [change_end] ------------------------------------------------------------------------------- 23.1.5.3. Using Logical Pathnames [change_begin] Some real file systems do not have versions. Logical pathname translation to such a file system ignores the version. This implies that a portable program cannot rely on being able to store in a file system more than one version of a file named by a logical pathname. The type of a logical pathname for a Common Lisp source file is LISP. This should be translated into whatever implementation-defined type is appropriate in a physical pathname. The logical pathname host name SYS is reserved for the implementation. The existence and meaning of logical pathnames for logical host SYS is implementation-defined. File manipulation functions must operate with logical pathnames according to the following requirements: * The following accept logical pathnames and translate them into physical pathnames as if by calling the function translate-logical-pathname: compile-file ed probe-file compile-file-pathname file-author rename-file delete-file file-write-date truename directory load with-open-file dribble open * Applying the function pathname to a stream created by the function open or the macro with-open-file using a logical pathname produces a logical pathname. * The functions truename, probe-file, and directory never return logical pathnames. * Calling rename-file with a logical pathname as the second argument returns a logical pathname as the first value. * make-pathname returns a logical pathname if and only if the host is logical. If the :host argument to make-pathname is supplied, the host is logical if it came from the pathname-host of a logical pathname. Whether a :host argument is logical if it is a string equal to a logical pathname host name is implementation-defined. [Function] logical-pathname pathname Converts the argument to a logical pathname and returns it. The argument can be a logical pathname, a logical pathname namestring containing a host component, or a stream for which the pathname function returns a logical pathname. For any other argument, logical-pathname signals an error of type type-error. [Function] translate-logical-pathname pathname &key Translates a logical pathname to the corresponding physical pathname. The pathname argument is first coerced to a pathname. If it is not a pathname, string, or file stream, an error of type type-error is signaled. If the coerced argument is a physical pathname, it is returned. If the coerced argument is a logical pathname, the first matching translation (according to pathname-match-p) of the logical pathname host is applied, as if by calling translate-pathname. If the result is a logical pathname, this process is repeated. When the result is finally a physical pathname, it is returned. If no translation matches a logical pathname, an error of type file-error is signaled. translate-logical-pathname may perform additional translations, typically to provide translation of file types to local naming conventions, to accommodate physical file systems with names of limited length, or to deal with special character requirements such as translating hyphens to underscores or uppercase letters to lowercase. Any such additional translations are implementation-defined. Some implementations do no additional translations. There are no specified keyword arguments for translate-logical-pathname but implementations are permitted to extend it by adding keyword arguments. There is one specified return value from translate-logical-pathname; implementations are permitted to extend it by returning additional values. [Function] logical-pathname-translations host If the specified host is not the host component of a logical pathname and is not a string that has been defined as a logical pathname host name by setf of logical-pathname-translations, this function signals an error of type type-error; otherwise, it returns the list of translations for the specified host. Each translation is a list of at least two elements, from-wildname and to-wildname. Any additional elements are implementation-defined. A from-wildname is a logical pathname whose host is the specified host. A to-wildname is any pathname. Translations are searched in the order listed, so more specific from-wildnames must precede more general ones. (setf (logical-pathname-translations host) translations) sets the list of translations for the logical pathname host to translations. If host is a string that has not previously been used as logical pathname host, a new logical pathname host is defined; otherwise an existing host's translations are replaced. Logical pathname host names are compared with string-equal. When setting the translations list, each from-wildname can be a logical pathname whose host is host or a logical pathname namestring s parseable by (parse-namestring s host-object), where host-object is an appropriate object for representing the specified host to parse-namestring. (This circuitous specification dodges the fact that parse-namestring does not necessarily accept as its second argument any old string that names a logical host.) Each to-wildname can be anything coercible to a pathname by application of the function pathname. If to-wildname coerces to a logical pathname, translate-logical-pathname will retranslate the result, repeatedly if necessary. Implementations may define additional functions that operate on logical pathname hosts (for example, to specify additional translation rules or options). [Function] load-logical-pathname-translations host If a logical pathname host named host (a string) is already defined, this function returns nil. Otherwise, it searches for a logical pathname host definition in an implementation-defined manner. If none is found, it signals an error. If a definition is found, it installs the definition and returns t. The search used by load-logical-pathname-translations should be documented, as logical pathname definitions will be created by users as well as by Lisp implementors. A typical search technique is to look in an implementation-defined directory for a file whose name is derived from the host name in an implementation-defined fashion. [Function] compile-file-pathname pathname &key :output-file Returns the pathname that compile-file would write into, if given the same arguments. If the pathname argument is a logical pathname and the :output-file argument is unspecified, the result is a logical pathname. If an implementation supports additional keyword arguments to compile-file, compile-file-pathname must accept the same arguments. [change_end] ------------------------------------------------------------------------------- 23.1.5.4. Examples of the Use of Logical Pathnames [change_begin] Here is a very simple example of setting up a logical pathname host named FOO. Suppose that no translations are necessary to get around file system restrictions, so all that is necessary is to specify the root of the physical directory tree that contains the logical file system. The namestring syntax in the to-wildname is implementation-specific. (setf (logical-pathname-translations "foo") '(("**;*.*.*" "MY-LISPM:>library>foo>**>"))) The following is a sample use of that logical pathname. All return values are of course implementation-specific; all of the examples in this section are of course meant to be illustrative and not prescriptive. (translate-logical-pathname "foo:bar;baz;mum.quux.3") => #P"MY-LISPM:>library>foo>bar>baz>mum.quux.3" Next we have a more complex example, dividing the files among two file servers (U, supporting a UNIX file system, and V, supporting a VAX/VMS file system) and several different directories. This UNIX file system doesn't support :wild-inferiors in the directory, so each directory level must be translated individually. No file name or type translations are required except for .MAIL to .MBX. The namestring syntax used for the to-wildnames is implementation-specific. (setf (logical-pathname-translations "prog") '(("RELEASED;*.*.*" "U:/sys/bin/my-prog/") ("RELEASED;*;*.*.*" "U:/sys/bin/my-prog/*/") ("EXPERIMENTAL;*.*.*" "U:/usr/Joe/development/prog/") ("EXPERIMENTAL;DOCUMENTATION;*.*.*" "V:SYS$DISK:[JOE.DOC]") ("EXPERIMENTAL;*;*.*.*" "U:/usr/Joe/development/prog/*/") ("MAIL;**;*.MAIL" "V:SYS$DISK:[JOE.MAIL.PROG...]*.MBX") )) Here are sample uses of logical host PROG. All return values are of course implementation-specific. (translate-logical-pathname "prog:mail;save;ideas.mail.3") => #P"V:SYS$DISK:[JOE.MAIL.PROG.SAVE]IDEAS.MBX.3" (translate-logical-pathname "prog:experimental;spreadsheet.c") => #P"U:/usr/Joe/development/prog/spreadsheet.c" Suppose now that we have a program that uses three files logically named MAIN.LISP, AUXILIARY.LISP, and DOCUMENTATION.LISP. The following translations might be provided by a software supplier as examples. For a UNIX file system with long file names: (setf (logical-pathname-translations "prog") '(("CODE;*.*.*" "/lib/prog/"))) (translate-logical-pathname "prog:code;documentation.lisp") => #P"/lib/prog/documentation.lisp" For a UNIX file system with 14-character file names, using .lisp as the type: (setf (logical-pathname-translations "prog") '(("CODE;DOCUMENTATION.*.*" "/lib/prog/docum.*") ("CODE;*.*.*" "/lib/prog/"))) (translate-logical-pathname "prog:code;documentation.lisp") => #P"/lib/prog/docum.lisp" For a UNIX file system with 14-character file names, using .l as the type (the second translation shortens the compiled file type to .b): (setf (logical-pathname-translations "prog") `(("**;*.LISP.*" ,(logical-pathname "PROG:**;*.L.*")) (,(compile-file-pathname (logical-pathname "PROG:**;*.LISP.*")) ,(logical-pathname "PROG:**;*.B.*")) ("CODE;DOCUMENTATION.*.*" "/lib/prog/documentatio.*") ("CODE;*.*.*" "/lib/prog/"))) (translate-logical-pathname "prog:code;documentation.lisp") => #P"/lib/prog/documentatio.l" [change_end] ------------------------------------------------------------------------------- 23.1.5.5. Discussion of Logical Pathnames [change_begin] Large programs can be moved between sites without changing any pathnames, provided all pathnames used are logical. A portable system construction tool can be created that operates on programs defined as sets of files named by logical pathnames. Logical pathname syntax was chosen to be easily translated into the formats of most popular file systems, while still being powerful enough for storing large programs. Although they have hierarchical directories, extended wildcard matching, versions, and no limit on the length of names, logical pathnames can be mapped onto a less capable real file system by translating each directory that is used into a flat directory name, processing wildcards in the Lisp implementation rather than in the file system, treating all versions as :newest, and using translations to shorten long names. Logical pathname words are restricted to non-case-sensitive letters, digits, and hyphens to avoid creating problems with real file systems that support limited character sets for file naming. (If logical pathnames were case-sensitive, it would be very difficult to map them into a file system that is not sensitive to case in its file names.) It is not a goal of logical pathnames to be able to represent all possible file names. Their goal is rather to represent just enough file names to be useful for storing software. Real pathnames, in contrast, need to provide a uniform interface to all possible file names, including names and naming conventions that are not under the control of Common Lisp. The choice of logical pathname syntax, using colon, semicolon, and period, was guided by the goals of being visually distinct from real file systems and minimizing the use of special characters. The logical-pathname function is separate from the pathname function so that the syntax of logical pathname namestrings does not constrain the syntax of physical pathname namestrings in any way. Logical pathname syntax must be defined by Common Lisp so that logical pathnames can be conveniently exchanged between implementations, but physical pathname syntax is dictated by the operating environments. The compile-file-pathname function and the specification of LISP as the type of a logical pathname for a Common Lisp source file together provide enough information about compilation to make possible a portable system construction tool. Suppose that it is desirable to call compile-file only if the source file is newer than the compiled file. For this to succeed, it must be possible to know the name of the compiled file without actually calling compile-file. In some implementations the compiler produces one of several file types, depending on a variety of implementation-dependent circumstances, so it is not sufficient simply to prescribe a standard logical file type for compiled files; compile-file-pathname provides access to the defaulting that is performed by compile-file ``in a manner appropriate to the implementation's file system conventions.'' The use of the logical pathname host name SYS for the implementation is current practice. Standardizing on this name helps users choose logical pathname host names that avoid conflicting with implementation-defined names. Loading of logical pathname translations from a site-dependent file allows software to be distributed using logical pathnames. The assumed model of software distribution is a division of labor between the supplier of the software and the user installing it. The supplier chooses logical pathnames to name all the files used or created by the software, and supplies examples of logical pathname translations for a few popular file systems. Each example uses an assumed directory and/or device name, assumes local file naming conventions, and provides translations that will translate all the logical pathnames used or generated by the particular software into valid physical pathnames. For a powerful file system these translations can be quite simple. For a more restricted file system, it may be necessary to list an explicit translation for every logical pathname used (for example, when dealing with restrictions on the maximum length of a file name). The user installing the software decides on which device and directory to store the files and edits the example logical pathname translations accordingly. If necessary, the user also adjusts the translations for local file naming conventions and any other special aspects of the user's local file system policy and local Common Lisp implementation. For example, the files might be divided among several file server hosts to share the load. The process of defining site-customized logical pathname translations is quite easy for a user of a popular file system for which the software supplier has provided an example. A user of a more unusual file system might have to take more time; the supplier can help by providing a list of all the logical pathnames used or generated by the software.Once the user has created and executed a suitable setf form for setting the logical-pathname-translations of the relevant logical host, the software can be loaded and run. It may be necessary to use the translations again, or on another workstation at the same site, so it is best to save the setf form in the standard place where it can be found later by load-logical-pathname-translations. Often a software supplier will include a program for restoring software from the distribution medium to the file system and a program for loading the software from the file system into a Common Lisp; these programs will start by calling load-logical-pathname-translations to make sure that the logical pathname host is defined. Note that the setf of logical-pathname-translations form isn't part of the program; it is separate and is written by the user, not by the software supplier. That separation and a uniform convention for doing the separation are the key aspects of logical pathnames. For small programs involving only a handful of files, it doesn't matter much. The real benefits come with large programs with hundreds or thousands of files and more complicated situations such as program-generated file names or porting a program developed on a system with long file names onto a system with a very restrictive limit on the length of file names. [change_end] ------------------------------------------------------------------------------- 23.1.6. Pathname Functions These functions are what programs use to parse and default file names that have been typed in or otherwise supplied by the user. [old_change_begin] Any argument called pathname in this book may actually be a pathname, a string or symbol, or a stream. Any argument called defaults may likewise be a pathname, a string or symbol, or a stream. [old_change_end] [change_begin] X3J13 voted in March 1988 (PATHNAME-SYMBOL) to change the language so that a symbol is never allowed as a pathname argument. More specifically, the following functions are changed to disallow a symbol as a pathname argument: pathname pathname-device namestring truename pathname-directory file-namestring parse-namestring pathname-name directory-namestring merge-pathnames pathname-type host-namestring pathname-host pathname-version enough-namestring (The function require was also changed by this vote but was deleted from the language by a vote in January 1989 (REQUIRE-PATHNAME-DEFAULTS) .) Furthermore, the vote reaffirmed that the following functions do not accept symbols as file, filename, or pathname arguments: open rename-file file-write-date with-open-file delete-file file-author load probe-file directory compile-file In older implementations of Lisp that did not have strings, for example MacLisp, symbols were the only means for specifying pathnames. This was convenient only because the file systems of the time allowed only uppercase letters in file names. Typing (load 'foo) caused the function load to receive the symbol FOO (with uppercase letters because of the way symbols are parsed) and therefore to load the file named FOO. Now that many file systems, most notably UNIX, support case-sensitive file names, the use of symbols is less convenient and more error-prone. X3J13 voted in March 1988 (PATHNAME-STREAM) to specify that a stream may be used as a pathname, file, or filename argument only if it was created by use of open or with-open-file, or if it is a synonym stream whose symbol is bound to a stream that may be used as a pathname. If such a stream is used as a pathname, it is as if the pathname function were applied to the stream and the resulting pathname used in place of the stream. This represents the name used to open the file. This may be, but is not required to be, the actual name of the file. It is an error to attempt to obtain a pathname from a stream created by any of the following: make-two-way-stream make-string-input-stream make-echo-stream make-string-output-stream make-broadcast-stream with-input-from-string make-concatenated-stream with-output-to-string [change_end] In the examples, it is assumed that the host named CMUC runs the TOPS-20 operating system, and therefore uses TOPS-20 file system syntax; furthermore, an explicit host name is indicated by following the host name with a double colon. Remember, however, that namestring syntax is implementation-dependent, and this syntax is used here purely for the sake of examples. [Function] pathname pathname The pathname function converts its argument to be a pathname. The argument may be a pathname, a string or symbol, or a stream; the result is always a pathname. [change_begin] X3J13 voted in March 1988 not to permit symbols as pathnames (PATHNAME-SYMBOL) and to specify exactly which streams may be used as pathnames (PATHNAME-STREAM) . X3J13 voted in January 1989 (CLOSED-STREAM-OPERATIONS) to specify that pathname is unaffected by whether its argument, if a stream, is open or closed. X3J13 further commented that because some implementations cannot provide the ``true name'' of a file until the file is closed, in such an implementation pathname might, in principle, return a different (perhaps more specific) file name after the stream is closed. However, such behavior is prohibited; pathname must return the same pathname after a stream is closed as it would have while the stream was open. See truename. [change_end] [Function] truename pathname The truename function endeavors to discover the ``true name'' of the file associated with the pathname within the file system. If the pathname is an open stream already associated with a file in the file system, that file is used. The ``true name'' is returned as a pathname. An error is signaled if an appropriate file cannot be located within the file system for the given pathname. The truename function may be used to account for any file name translations performed by the file system, for example. For example, suppose that DOC: is a TOPS-20 logical device name that is translated by the TOPS-20 file system to be PS:<DOCUMENTATION>. (setq file (open "CMUC::DOC:DUMPER.HLP")) (namestring (pathname file)) => "CMUC::DOC:DUMPER.HLP" (namestring (truename file)) => "CMUC::PS:<DOCUMENTATION>DUMPER.HLP.13" [change_begin] X3J13 voted in March 1988 not to permit symbols as pathnames (PATHNAME-SYMBOL) and to specify exactly which streams may be used as pathnames (PATHNAME-STREAM) . X3J13 voted in January 1989 (CLOSED-STREAM-OPERATIONS) to specify that truename may be applied to a stream whether the stream is open or closed. X3J13 further commented that because some implementations cannot provide the ``true name'' of a file until the file is closed, in principle it would be possible in such an implementation for truename to return a different file name after the stream is closed. Such behavior is permitted; in this respect truename differs from pathname. X3J13 voted in June 1989 (PATHNAME-WILD) to clarify that truename accepts only non-wild pathnames; an error is signaled if wild-pathname-p would be true of the pathname argument. X3J13 voted in June 1989 (PATHNAME-LOGICAL) to require truename to accept logical pathnames (see section 23.1.5). However, truename never returns a logical pathname. [change_end] [Function] parse-namestring thing &optional host defaults &key :start :end :junk-allowed [old_change_begin] This turns thing into a pathname. The thing is usually a string (that is, a namestring), but it may be a symbol (in which case the print name is used) or a pathname or stream (in which case no parsing is needed, but an error check may be made for matching hosts). [old_change_end] [change_begin] X3J13 voted in March 1988 not to permit symbols as pathnames (PATHNAME-SYMBOL) and to specify exactly which streams may be used as pathnames (PATHNAME-STREAM) . The thing argument may not be a symbol. X3J13 voted in June 1989 (PATHNAME-LOGICAL) to require parse-namestring to accept logical pathname namestrings (see section 23.1.5). [change_end] This function does not, in general, do defaulting of pathname components, even though it has an argument named defaults; it only does parsing. The host and defaults arguments are present because in some implementations it may be that a namestring can only be parsed with reference to a particular file name syntax of several available in the implementation. If host is non-nil, it must be a host name that could appear in the host component of a pathname, or nil; if host is nil then the host name is extracted from the default pathname in defaults and used to determine the syntax convention. The defaults argument defaults to the value of *default-pathname-defaults*. For a string (or symbol) argument, parse-namestring parses a file name within it in the range delimited by the :start and :end arguments (which are integer indices into string, defaulting to the beginning and end of the string). [change_begin] See chapter 14 for a discussion of :start and :end arguments. [change_end] If :junk-allowed is not nil, then the first value returned is the pathname parsed, or nil if no syntactically correct pathname was seen. If :junk-allowed is nil (the default), then the entire substring is scanned. The returned value is the pathname parsed. An error is signaled if the substring does not consist entirely of the representation of a pathname, possibly surrounded on either side by whitespace characters if that is appropriate to the cultural conventions of the implementation. In either case, the second value is the index into the string of the delimiter that terminated the parse, or the index beyond the substring if the parse terminated at the end of the substring (as will always be the case if :junk-allowed is false). If thing is not a string or symbol, then start (which defaults to zero in any case) is always returned as the second value. Parsing an empty string always succeeds, producing a pathname with all components (except the host) equal to nil. Note that if host is specified and not nil, and thing contains a manifest host name, an error is signaled if the hosts do not match. If thing contains an explicit host name and no explicit device name, then it might be appropriate, depending on the implementation environment, for parse-namestring to supply the standard default device for that host as the device component of the resulting pathname. [Function] merge-pathnames pathname &optional defaults default-version [old_change_begin] This is the function that most programs should call to process a file name supplied by the user. It fills in unspecified components of pathname from the defaults, and returns a new pathname. The pathname and defaults arguments may each be a pathname, stream, string, or symbol. The result is always a pathname. [old_change_end] [change_begin] X3J13 voted in March 1988 not to permit symbols as pathnames (PATHNAME-SYMBOL) and to specify exactly which streams may be used as pathnames (PATHNAME-STREAM) . X3J13 voted in June 1989 (PATHNAME-LOGICAL) to require merge-namestrings to recognize a logical pathname namestring as its first argument if its second argument is a logical pathname (see section 23.1.5). X3J13 voted in January 1989 (CLOSED-STREAM-OPERATIONS) to specify that merge-pathname is unaffected by whether the first argument, if a stream, is open or closed. If the first argument is a stream, merge-pathname behaves as if the function pathname were applied to the stream and the resulting pathname used instead. X3J13 voted in June 1989 (PATHNAME-COMPONENT-CASE) to require merge-pathnames to map customary case in argument pathnames to the customary case in returned pathnames (see section 23.1.2). [change_end] defaults defaults to the value of *default-pathname-defaults*. default-version defaults to :newest. Here is an example of the use of merge-pathnames: (merge-pathnames "CMUC::FORMAT" "CMUC::PS:<LISPIO>.FASL") => a pathname object that re-expressed as a namestring would be "CMUC::PS:<LISPIO>FORMAT.FASL.0" Defaulting of pathname components is done by filling in components taken from another pathname. This is especially useful for cases such as a program that has an input file and an output file, and asks the user for the name of both, letting the unsupplied components of one name default from the other. Unspecified components of the output pathname will come from the input pathname, except that the type should default not to the type of the input but to the appropriate default type for output from this program. The pathname merging operation takes as input a given pathname, a defaults pathname, and a default version, and returns a new pathname. Basically, the missing components in the given pathname are filled in from the defaults pathname, except that if no version is specified the default version is used. The default version is usually :newest; if no version is specified the newest version in existence should be used. The default version can be nil, to preserve the information that it was missing in the input pathname. If the given pathname explicitly specifies a host and does not supply a device, then if the host component of the defaults matches the host component of the given pathname, then the device is taken from the defaults; otherwise the device will be the default file device for that host. Next, if the given pathname does not specify a host, device, directory, name, or type, each such component is copied from the defaults. The merging rules for the version are more complicated and depend on whether the pathname specifies a name. If the pathname doesn't specify a name, then the version, if not provided, will come from the defaults, just like the other components. However, if the pathname does specify a name, then the version is not affected by the defaults. The reason is that the version ``belongs to'' some other file name and is unlikely to have anything to do with the new one. Finally, if this process leaves the version missing, the default version is used. The net effect is that if the user supplies just a name, then the host, device, directory, and type will come from the defaults, but the version will come from the default version argument to the merging operation. If the user supplies nothing, or just a directory, the name, type, and version will come over from the defaults together. If the host's file name syntax provides a way to input a version without a name or type, the user can let the name and type default but supply a version different from the one in the defaults. [change_begin] X3J13 voted in June 1989 (PATHNAME-SYNTAX-ERROR-TIME) to agree to disagree: merge-pathname might or might not perform plausibility checking on its arguments to ensure that the resulting pathname can be converted a valid namestring. User beware: this could cause portability problems. For example, suppose that host LOSER constrains file types to be three characters or fewer but host CMUC does not. Then "LOSER::FORMAT" is a valid namestring and "CMUC::PS:<LISPIO>.FASL" is a valid namestring, but (merge-pathnames "LOSER::FORMAT" "CMUC::PS:<LISPIO>.FASL") might signal an error in some implementations because the hypothetical result would be a pathname equivalent to the namestring "LOSER::FORMAT.FASL" which is illegal because the file type FASL has more than three characters. In other implementations merge-pathname might return a pathname but that pathname might cause namestring to signal an error. [change_end] [Variable] *default-pathname-defaults* This is the default pathname-defaults pathname; if any pathname primitive that needs a set of defaults is not given one, it uses this one. As a general rule, however, each program should have its own pathname defaults rather than using this one. [old_change_begin] [Function] make-pathname &key :host :device :directory :name :type :version :defaults Given some components, make-pathname constructs and returns a pathname. After the components specified explicitly by the :host, :device, :directory, :name, :type, and :version arguments are filled in, the merging rules used by merge-pathnames are used to fill in any missing components from the defaults specified by the :defaults argument. The default value of the :defaults argument is a pathname whose host component is the same as the host component of the value of *default-pathname-defaults*, and whose other components are all nil. Whenever a pathname is constructed, whether by make-pathname or some other function, the components may be canonicalized if appropriate. For example, if a file system is insensitive to case, then alphabetic characters may be forced to be all uppercase or all lowercase by the implementation. [old_change_end] [change_begin] The following example assumes the use of UNIX syntax and conventions. (make-pathname :host "technodrome" :directory '(:absolute "usr" "krang") :name "shredder") => #P"technodrome:/usr/krang/shredder" X3J13 voted in June 1989 (PATHNAME-COMPONENT-CASE) to add a new keyword argument :case to make-pathname. The new argument description is therefore as follows: [Function] make-pathname &key :host :device :directory :name :type :version :defaults :case See section 23.1.2 for a description of the :case argument. X3J13 voted in June 1989 (PATHNAME-SYNTAX-ERROR-TIME) to agree to disagree: make-pathname might or might not check on its arguments to ensure that the resulting pathname can be converted to a valid namestring. If make-pathname does not check its arguments and signal an error in problematical cases, namestring yet might or might not signal an error when given the resulting pathname. User beware: this could cause portability problems. [change_end] [Function] pathnamep object This predicate is true if object is a pathname, and otherwise is false. (pathnamep x) == (typep x 'pathname) [old_change_begin] [Function] pathname-host pathname pathname-device pathname pathname-directory pathname pathname-name pathname pathname-type pathname pathname-version pathname These return the components of the argument pathname, which may be a pathname, string or symbol, or stream. The returned values can be strings, special symbols, or some other object in the case of structured components. The type will always be a string or a symbol. The version will always be a number or a symbol. [old_change_end] [change_begin] X3J13 voted in March 1988 not to permit symbols as pathnames (PATHNAME-SYMBOL) and to specify exactly which streams may be used as pathnames (PATHNAME-STREAM) . X3J13 voted in January 1989 (CLOSED-STREAM-OPERATIONS) to specify that these operations are unaffected by whether the first argument, if a stream, is open or closed. If the first argument is a stream, each operation behaves as if the function pathname were applied to the stream and the resulting pathname used instead. X3J13 voted in June 1989 (PATHNAME-COMPONENT-CASE) to add a keyword argument :case to all of the pathname accessor functions except pathname-version. The new argument descriptions are therefore as follows: [Function] pathname-host pathname &key :case pathname-device pathname &key :case pathname-directory pathname &key :case pathname-name pathname &key :case pathname-type pathname &key :case pathname-version pathname See section 23.1.2 for a description of the :case argument. X3J13 voted in June 1989 (PATHNAME-SUBDIRECTORY-LIST) to specify that pathname-directory always returns nil, :unspecific, or a list-never a string, never :wild (see section 23.1.3). If a list is returned, it is not guaranteed to be freshly consed; the consequences of modifying this list are undefined. [change_end] [Function] namestring pathname file-namestring pathname directory-namestring pathname host-namestring pathname enough-namestring pathname &optional defaults The pathname argument may be a pathname, a string or symbol, or a stream that is or was open to a file. The name represented by pathname is returned as a namelist in canonical form. If pathname is a stream, the name returned represents the name used to open the file, which may not be the actual name of the file (see truename). [change_begin] X3J13 voted in March 1988 not to permit symbols as pathnames (PATHNAME-SYMBOL) and to specify exactly which streams may be used as pathnames (PATHNAME-STREAM) . X3J13 voted in January 1989 (CLOSED-STREAM-OPERATIONS) to specify that these operations are unaffected by whether the first argument, if a stream, is open or closed. If the first argument is a stream, each operation behaves as if the function pathname were applied to the stream and the resulting pathname used instead. [change_end] namestring returns the full form of the pathname as a string. file-namestring returns a string representing just the name, type, and version components of the pathname; the result of directory-namestring represents just the directory-name portion; and host-namestring returns a string for just the host-name portion. Note that a valid namestring cannot necessarily be constructed simply by concatenating some of the three shorter strings in some order. enough-namestring takes another argument, defaults. It returns an abbreviated namestring that is just sufficient to identify the file named by pathname when considered relative to the defaults (which defaults to the value of *default-pathname-defaults*). That is, it is required that (merge-pathnames (enough-namestring pathname defaults) defaults) == (merge-pathnames (parse-namestring pathname nil defaults) defaults) in all cases; and the result of enough-namestring is, roughly speaking, the shortest reasonable string that will still satisfy this criterion. [change_begin] X3J13 voted in June 1989 (PATHNAME-SYNTAX-ERROR-TIME) to agree to disagree: make-pathname and merge-pathnames might or might not be able to produce pathnames that cannot be converted to valid namestrings. User beware: this could cause portability problems. [change_end] [Function] user-homedir-pathname &optional host Returns a pathname for the user's ``home directory'' on host. The host argument defaults in some appropriate implementation-dependent manner. The concept of ``home directory'' is itself somewhat implementation-dependent, but from the point of view of Common Lisp it is the directory where the user keeps personal files such as initialization files and mail. If it is impossible to determine this information, then nil is returned instead of a pathname; however, user-homedir-pathname never returns nil if the host argument is not specified. This function returns a pathname without any name, type, or version component (those components are all nil). ------------------------------------------------------------------------------- 23.2. Opening and Closing Files When a file is opened, a stream object is constructed to serve as the file system's ambassador to the Lisp environment; operations on the stream are reflected by operations on the file in the file system. The act of closing the file (actually, the stream) ends the association; the transaction with the file system is terminated, and input/output may no longer be performed on the stream. The stream function close may be used to close a file; the functions described below may be used to open them. The basic operation is open, but with-open-file is usually more convenient for most applications. [Function] open filename &key :direction :element-type :if-exists :if-does-not-exist :external-format [change_begin] X3J13 voted in June 1989 (MORE-CHARACTER-PROPOSAL) to add to the function open a new keyword argument :external-format. This argument did not appear in the preceding argument description in the first edition. [change_end] This returns a stream that is connected to the file specified by filename. The filename is the name of the file to be opened; it may be a string, a pathname, or a stream. (If the filename is a stream, then it is not closed first or otherwise affected; it is used merely to provide a file name for the opening of a new stream.) [change_begin] X3J13 voted in January 1989 (STREAM-ACCESS) to specify that the result of open, if it is a stream, is always a stream of type file-stream. X3J13 voted in March 1988 (PATHNAME-STREAM) to specify exactly which streams may be used as pathnames. See section 23.1.6. X3J13 voted in January 1989 (CLOSED-STREAM-OPERATIONS) to specify that open is unaffected by whether the first argument, if a stream, is open or closed. If the first argument is a stream, open behaves as if the function pathname were applied to the stream and the resulting pathname used instead. X3J13 voted in June 1989 (PATHNAME-WILD) to clarify that open accepts only non-wild pathnames; an error is signaled if wild-pathname-p would be true of filename. X3J13 voted in June 1989 (PATHNAME-LOGICAL) to require open to accept logical pathnames (see section 23.1.5). [change_end] The keyword arguments specify what kind of stream to produce and how to handle errors: :direction This argument specifies whether the stream should handle input, output, or both. :input The result will be an input stream. This is the default. :output The result will be an output stream. :io The result will be a bidirectional stream. :probe The result will be a no-directional stream (in effect, the stream is created and then closed). This is useful for determining whether a file exists without actually setting up a complete stream. :element-type This argument specifies the type of the unit of transaction for the stream. Anything that can be recognized as being a finite subtype of character or integer is acceptable. In particular, the following types are recognized: [old_change_begin] string-char The unit of transaction is a string-character. The functions read-char and/or write-char may be used on the stream. This is the default. character The unit of transaction is any character, not just a string-character. The functions read-char and/or write-char may be used on the stream. [old_change_end] [change_begin] X3J13 voted in June 1989 (MORE-CHARACTER-PROPOSAL) to eliminate the type string-char, add the type base-character, and redefine open to use the type character as the default :element-type. The preceding two possibilities should therefore be replaced by the following. character The unit of transaction is any character, not just a string-character. The functions read-char and write-char (depending on the value of the :direction argument) may be used on the stream. This is the default. base-character The unit of transaction is a base character. The functions read-char and write-char (depending on the value of the :direction argument) may be used on the stream. [change_end] (unsigned-byte n) The unit of transaction is an unsigned byte (a non-negative integer) of size n. The functions read-byte and/or write-byte may be used on the stream. unsigned-byte The unit of transaction is an unsigned byte (a non-negative integer); the size of the byte is determined by the file system. The functions read-byte and/or write-byte may be used on the stream. (signed-byte n) The unit of transaction is a signed byte of size n. The functions read-byte and/or write-byte may be used on the stream. signed-byte The unit of transaction is a signed byte; the size of the byte is determined by the file system. The functions read-byte and/or write-byte may be used on the stream. bit The unit of transaction is a bit (values 0 and 1). The functions read-byte and/or write-byte may be used on the stream. (mod n) The unit of transaction is a non-negative integer less than n. The functions read-byte and/or write-byte may be used on the stream. :default The unit of transaction is to be determined by the file system, based on the file it finds. The type can be determined by using the function stream-element-type. :if-exists This argument specifies the action to be taken if the :direction is :output or :io and a file of the specified name already exists. If the direction is :input or :probe, this argument is ignored. :error Signals an error. This is the default when the version component of the filename is not :newest. :new-version Creates a new file with the same file name but with a larger version number. This is the default when the version component of the filename is :newest. :rename Renames the existing file to some other name and then creates a new file with the specified name. :rename-and-delete Renames the existing file to some other name and then deletes it (but does not expunge it, on those systems that distinguish deletion from expunging). Then create a new file with the specified name. :overwrite Uses the existing file. Output operations on the stream will destructively modify the file. If the :direction is :io, the file is opened in a bidirectional mode that allows both reading and writing. The file pointer is initially positioned at the beginning of the file; however, the file is not truncated back to length zero when it is opened. This mode is most useful when the file-position function can be used on the stream. :append Uses the existing file. Output operations on the stream will destructively modify the file. The file pointer is initially positioned at the end of the file. If the :direction is :io, the file is opened in a bidirectional mode that allows both reading and writing. :supersede Supersedes the existing file. If possible, the implementation should arrange not to destroy the old file until the new stream is closed, against the possibility that the stream will be closed in ``abort'' mode (see close). This differs from :new-version in that :supersede creates a new file with the same name as the old one, rather than a file name with a higher version number. nil Does not create a file or even a stream, but instead simply returns nil to indicate failure. If the :direction is :output or :io and the value of :if-exists is :new-version, then the version of the (newly created) file that is opened will be a version greater than that of any other file in the file system whose other pathname components are the same as those of filename. If the :direction is :input or :probe or the value of :if-exists is not :new-version, and the version component of the filename is :newest, then the file opened is that file already existing in the file system that has a version greater than that of any other file in the file system whose other pathname components are the same as those of filename. [change_begin] Some file systems permit yet other actions to be taken when a file already exists; therefore, some implementations provide implementation-specific :if-exist options. [change_end] ------------------------------------------------------------------------------- Implementation note: The various file systems in existence today have widely differing capabilities. A given implementation may not be able to support all of these options in exactly the manner stated. An implementation is required to recognize all of these option keywords and to try to do something ``reasonable'' in the context of the host operating system. Implementors are encouraged to approximate the semantics specified here as closely as possible. As an example, suppose that a file system does not support distinct file versions and does not distinguish the notions of deletion and expunging (in some file systems file deletion is reversible until an expunge operation is performed). Then :new-version might be treated the same as :rename or :supersede, and :rename-and-delete might be treated the same as :supersede. If it is utterly impossible for an implementation to handle some option in a manner close to what is specified here, it may simply signal an error. The opening of files is an area where complete portability is too much to hope for; the intent here is simply to make things as portable as possible by providing specific names for a range of commonly supportable options. ------------------------------------------------------------------------------- :if-does-not-exist This argument specifies the action to be taken if a file of the specified name does not already exist. :error Signals an error. This is the default if the :direction is :input, or if the :if-exists argument is :overwrite or :append. :create Creates an empty file with the specified name and then proceeds as if it had already existed (but do not perform any processing directed by the :if-exists argument). This is the default if the :direction is :output or :io, and the :if-exists argument is anything but :overwrite or :append. nil Does not create a file or even a stream, but instead simply returns nil to indicate failure. This is the default if the :direction is :probe. [change_begin] X3J13 voted in June 1989 (MORE-CHARACTER-PROPOSAL) to add to the function open a new keyword argument :external-format. :external-format This argument specifies an implementation-recognized scheme for representing characters in files. The default value is :default and is implementation-defined but must support the base characters. An error is signaled if the implementation does recognize the specified format. This argument may be specified if the :direction argument is :input, :output, or :io. It is an error to write a character to the resulting stream that cannot be represented by the specified file format. (However, the #\Newline character cannot produce such an error; implementations must provide appropriate line division behavior for all character streams.) See stream-external-format. [change_end] When the caller is finished with the stream, it should close the file by using the close function. The with-open-file form does this automatically, and so is preferred for most purposes. open should be used only when the control structure of the program necessitates opening and closing of a file in some way more complex than provided by with-open-file. It is suggested that any program that uses open directly should use the special form unwind-protect to close the file if an abnormal exit occurs. [Macro] with-open-file (stream filename {options}*) {declaration}* {form}* with-open-file evaluates the forms of the body (an implicit progn) with the variable stream bound to a stream that reads or writes the file named by the value of filename. The options are evaluated and are used as keyword arguments to the function open. When control leaves the body, either normally or abnormally (such as by use of throw), the file is automatically closed. If a new output file is being written, and control leaves abnormally, the file is aborted and the file system is left, so far as possible, as if the file had never been opened. Because with-open-file always closes the file, even when an error exit is taken, it is preferred over open for most applications. filename is the name of the file to be opened; it may be a string, a pathname, or a stream. [change_begin] X3J13 voted in March 1988 (PATHNAME-STREAM) to specify exactly which streams may be used as pathnames. See section 23.1.6. X3J13 voted in June 1989 (PATHNAME-WILD) to clarify that with-open-file accepts only non-wild pathnames; an error is signaled if wild-pathname-p would be true of the filename argument. X3J13 voted in June 1989 (PATHNAME-LOGICAL) to require with-open-file to accept logical pathnames (see section 23.1.5). [change_end] For example: (with-open-file (ifile name :direction :input) (with-open-file (ofile (merge-pathname-defaults ifile nil "out") :direction :output :if-exists :supersede) (transduce-file ifile ofile))) [change_begin] X3J13 voted in June 1989 (WITH-OPEN-FILE-DOES-NOT-EXIST) to specify that the variable stream is not always bound to a stream; rather it is bound to whatever would be returned by a call to open. For example, if the options include :if-does-not-exist nil, stream will be bound to nil if the file does not exist. In this case the value of stream should be tested within the body of the with-open-file form before it is used as a stream. For example: (with-open-file (ifile name :direction :input :if-does-not-exist nil) ;; Process the file only if it actually exists. (when (streamp name) (compile-cobol-program ifile))) [change_end] ------------------------------------------------------------------------------- Implementation note: While with-open-file tries to automatically close the stream on exit from the construct, for robustness it is helpful if the garbage collector can detect discarded streams and automatically close them. ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- 23.3. Renaming, Deleting, and Other File Operations These functions provide a standard interface to operations provided in some form by most file systems. It may be that some implementations of Common Lisp cannot support them all completely. [Function] rename-file file new-name The specified file is renamed to new-name (which must be a file name). The file may be a string, a pathname, or a stream. If it is an open stream associated with a file, then the stream itself and the file associated with it are affected (if the file system permits). [change_begin] X3J13 voted in March 1988 (PATHNAME-STREAM) to specify exactly which streams may be used as pathnames. See section 23.1.6. [change_end] rename-file returns three values if successful. The first value is the new-name with any missing components filled in by performing a merge-pathnames operation using file as the defaults. The second value is the truename of the file before it was renamed. The third value is the truename of the file after it was renamed. If the renaming operation is not successful, an error is signaled. [old_change_begin] It is an error to specify a file name containing a :wild component, for file to contain a nil component where the file system does not permit a nil component, or for the result of defaulting missing components of new-name from file to contain a nil component where the file system does not permit a nil component. [old_change_end] [change_begin] X3J13 voted in June 1989 (PATHNAME-WILD) to specify that supplying a wild pathname as the file argument to rename-file has implementation-dependent consequences; rename-file might signal an error, for example, or might rename all files that match the wild pathname. X3J13 voted in June 1989 (PATHNAME-LOGICAL) to require rename-file to accept logical pathnames (see section 23.1.5). [change_end] ------------------------------------------------------------------------------- Compatibility note: This corresponds to the function called renamef in MacLisp and Lisp Machine Lisp. The name renamef is not used in Common Lisp because the convention that a trailing f means ``file'' conflicts with the use of a trailing f for forms related to setf. ------------------------------------------------------------------------------- [Function] delete-file file The specified file is deleted. The file may be a string, a pathname, or a stream. If it is an open stream associated with a file, then the stream itself and the file associated with it are affected (if the file system permits), in which case the stream may or may not be closed immediately, and the deletion may be immediate or delayed until the stream is explicitly closed, depending on the requirements of the file system. [change_begin] X3J13 voted in March 1988 (PATHNAME-STREAM) to specify exactly which streams may be used as pathnames. See section 23.1.6. [change_end] [old_change_begin] delete-file returns a non-nil value if successful. It is left to the discretion of the implementation whether an attempt to delete a non-existent file is considered to be successful. If the deleting operation is not successful, an error is signaled. It is an error to specify a file name that contains a :wild component or one that contains a nil component where the file system does not permit a nil component. [old_change_end] [change_begin] X3J13 voted in June 1989 (PATHNAME-WILD) to clarify that supplying a wild pathname as the file argument to delete-file has implementation-dependent consequences; delete-file might signal an error, for example, or might delete all files that match the wild pathname. X3J13 voted in June 1989 (PATHNAME-LOGICAL) to require delete-file to accept logical pathnames (see section 23.1.5). [change_end] ------------------------------------------------------------------------------- Compatibility note: This corresponds to the function called deletef in MacLisp and Lisp Machine Lisp. ------------------------------------------------------------------------------- [Function] probe-file file This predicate is false if there is no file named file, and otherwise returns a pathname that is the true name of the file (which may be different from file because of file links, version numbers, or other artifacts of the file system). Note that if the file is an open stream associated with a file, then probe-file cannot return nil but will produce the true name of the associated file. See truename and the :probe value for the :direction argument to open. This corresponds to the function called probef in MacLisp and Lisp Machine Lisp. [change_begin] X3J13 voted in March 1988 (PATHNAME-STREAM) to specify exactly which streams may be used as pathnames. See section 23.1.6. X3J13 voted in June 1989 (PATHNAME-WILD) to clarify that probe-file accepts only non-wild pathnames; an error is signaled if wild-pathname-p would be true of the file argument. X3J13 voted in June 1989 (PATHNAME-LOGICAL) to require probe-file to accept logical pathnames (see section 23.1.5). However, probe-file never returns a logical pathname. X3J13 voted in January 1989 (CLOSED-STREAM-OPERATIONS) to specify that probe-file is unaffected by whether the first argument, if a stream, is open or closed. If the first argument is a stream, probe-file behaves as if the function pathname were applied to the stream and the resulting pathname used instead. However, X3J13 further commented that the treatment of open streams may differ considerably from one implementation to another; for example, in some operating systems open files are written under a temporary or invisible name and later renamed when closed. In general, programmers writing code intended to be portable should be very careful when using probe-file. [change_end] [Function] file-write-date file file can be a file name or a stream that is open to a file. This returns the time at which the file was created or last written as an integer in universal time format (see section 25.4.1), or nil if this cannot be determined. [change_begin] X3J13 voted in March 1988 (PATHNAME-STREAM) to specify exactly which streams may be used as pathnames. See section 23.1.6. X3J13 voted in June 1989 (PATHNAME-WILD) to clarify that file-write-date accepts only non-wild pathnames; an error is signaled if wild-pathname-p would be true of the file argument. X3J13 voted in June 1989 (PATHNAME-LOGICAL) to require file-write-date to accept logical pathnames (see section 23.1.5). [change_end] [Function] file-author file file can be a file name or a stream that is open to a file. This returns the name of the author of the file as a string, or nil if this cannot be determined. [change_begin] X3J13 voted in March 1988 (PATHNAME-STREAM) to specify exactly which streams may be used as pathnames. See section 23.1.6. X3J13 voted in June 1989 (PATHNAME-WILD) to clarify that file-author accepts only non-wild pathnames; an error is signaled if wild-pathname-p would be true of the file argument. X3J13 voted in June 1989 (PATHNAME-LOGICAL) to require file-author to accept logical pathnames (see section 23.1.5). [change_end] [Function] file-position file-stream &optional position file-position returns or sets the current position within a random-access file. (file-position file-stream) returns a non-negative integer indicating the current position within the file-stream, or nil if this cannot be determined. The file position at the start of a file will be zero. The value returned by file-position increases monotonically as input or output operations are performed. For a character file, performing a single read-char or write-char operation may cause the file position to be increased by more than 1 because of character-set translations (such as translating between the Common Lisp #\Newline character and an external ASCII carriage-return/line-feed sequence) and other aspects of the implementation. For a binary file, every read-byte or write-byte operation increases the file position by 1. (file-position file-stream position) sets the position within file-stream to be position. The position may be an integer, or :start for the beginning of the stream, or :end for the end of the stream. If the integer is too large or otherwise inappropriate, an error is signaled (the file-length function returns the length beyond which file-position may not access). An integer returned by file-position of one argument should, in general, be acceptable as a second argument for use with the same file. With two arguments, file-position returns t if the repositioning was performed successfully, or nil if it was not (for example, because the file was not random-access). ------------------------------------------------------------------------------- Implementation note: Implementations that have character files represented as a sequence of records of bounded size might choose to encode the file position as, for example, record-number*256+character-within-record. This is a valid encoding because it increases monotonically as each character is read or written, though not necessarily by 1 at each step. An integer might then be considered ``inappropriate'' as a second argument to file-position if, when decoded into record number and character number, it turned out that the specified record was too short for the specified character number. ------------------------------------------------------------------------------- Compatibility note: This corresponds to the function called filepos in MacLisp and Lisp Machine Lisp. ------------------------------------------------------------------------------- [Function] file-length file-stream file-stream must be a stream that is open to a file. The length of the file is returned as a non-negative integer, or nil if the length cannot be determined. For a binary file, the length is specifically measured in units of the :element-type specified when the file was opened (see open). ------------------------------------------------------------------------------- Compatibility note: This corresponds to the function called lengthf in MacLisp and Lisp Machine Lisp. ------------------------------------------------------------------------------- [change_begin] [Function] file-string-length file-stream object X3J13 voted in June 1989 (MORE-CHARACTER-PROPOSAL) to add the function file-string-length. The object must be a string or a character. The function file-string-length returns a non-negative integer that is the difference between what the file-position of the file-stream would be after and before writing the object to the file-stream, or nil if this difference cannot be determined. The value returned may depend on the current state of the file-stream; that is, calling file-string-length on the same arguments twice may in certain circumstances produce two different integers. [change_end] ------------------------------------------------------------------------------- 23.4. Loading Files To load a file is to read through the file, evaluating each form in it. Programs are typically stored in files containing calls to constructs such as defun, defmacro, and defvar, which define the functions and variables of the program. Loading a compiled (``fasload'') file is similar, except that the file does not contain text but rather pre-digested expressions created by the compiler that can be loaded more quickly. [Function] load filename &key :verbose :print :if-does-not-exist This function loads the file named by filename into the Lisp environment. It is assumed that a text (character file) can be automatically distinguished from an object (binary) file by some appropriate implementation-dependent means, possibly by the file type. The defaults for filename are taken from the variable *default-pathname-defaults*. If the filename (after the merging in of the defaults) does not explicitly specify a type, and both text and object types of the file are available in the file system, load should try to select the more appropriate file by some implementation-dependent means. If the first argument is a stream rather than a pathname, then load determines what kind of stream it is and loads directly from the stream. The :verbose argument (which defaults to the value of *load-verbose*), if true, permits load to print a message in the form of a comment (that is, with a leading semicolon) to *standard-output* indicating what file is being loaded and other useful information. [old_change_begin] The :print argument (default nil), if true, causes the value of each expression loaded to be printed to *standard-output*. If a binary file is being loaded, then what is printed may not reflect precisely the contents of the source file, but nevertheless some information will be printed. [old_change_end] [change_begin] X3J13 voted in March 1989 (COMPILER-VERBOSITY) to add the variable *load-print*; its value is used as the default for the :print argument to load. The function load rebinds *package* to its current value. If some form in the file changes the value of *package* during loading, the old value will be restored when the loading is completed. (This was specified in the first edition under the description of *package*; for convenience I now mention it here as well.) X3J13 voted in March 1988 (PATHNAME-STREAM) to specify exactly which streams may be used as pathnames. See section 23.1.6. X3J13 voted in June 1989 (PATHNAME-WILD) to clarify that supplying a wild pathname as the filename argument to load has implementation-dependent consequences; load might signal an error, for example, or might load all files that match the pathname. X3J13 voted in June 1989 (PATHNAME-LOGICAL) to require load to accept logical pathnames (see section 23.1.5). [change_end] If a file is successfully loaded, load always returns a non-nil value. If :if-does-not-exist is specified and is nil, load just returns nil rather than signaling an error if the file does not exist. [change_begin] X3J13 voted in March 1989 (IN-SYNTAX) to require that load bind *readtable* to its current value at the time load is called; the dynamic extent of the binding should encompass all of the file-loading activity. This allows a portable program to include forms such as (in-package "FOO") (eval-when (:execute :load-toplevel :compile-toplevel) (setq *readtable* foo:my-readtable)) without performing a net global side effect on the loading environment. Such statements allow the remainder of such a file to be read either as interpreted code or by compile-file in a syntax determined by an alternative readtable. X3J13 voted in June 1989 (LOAD-TRUENAME) to require that load bind two new variables *load-pathname* and *load-truename*; the dynamic extent of the bindings should encompass all of the file-loading activity. [change_end] [Variable] *load-verbose* This variable provides the default for the :verbose argument to load. Its initial value is implementation-dependent. [change_begin] [Variable] *load-print* X3J13 voted in March 1989 (COMPILER-VERBOSITY) to add *load-print*. This variable provides the default for the :print argument to load. Its initial value is nil. [Variable] *load-pathname* X3J13 voted in June 1989 (LOAD-TRUENAME) to introduce *load-pathname*; it is initially nil but load binds it to a pathname that represents the file name given as the first argument to load merged with the defaults (see merge-pathname). [Variable] *load-truename* X3J13 voted in June 1989 (LOAD-TRUENAME) to introduce *load-truename*; it is initially nil but load binds it to the ``true name'' of the file being loaded. See truename. X3J13 voted in March 1989 (LOAD-OBJECTS) to introduce a facility based on the Object System whereby a user can specify how compile-file and load must cooperate to reconstruct compile-time constant objects at load time. The protocol is simply this: compile-file calls the generic function make-load-form on any object that is referenced as a constant or as a self-evaluating form, if the object's metaclass is standard-class, structure-class, any user-defined metaclass (not a subclass of built-in-class), or any of a possibly empty implementation-defined list of other metaclasses; compile-file will call make-load-form only once for any given object (as determined by eq) within a single file. The user-programmability stems from the possibility of user-defined methods for make-load-form. The helper function make-load-form-saving-slots makes it easy to write commonly used versions of such methods. [Generic function] make-load-form object The argument is an object that is referenced as a constant or as a self-evaluating form in a file being compiled by compile-file. The objective is to enable load to construct an equivalent object. The first value, called the creation form, is a form that, when evaluated at load time, should return an object that is equivalent to the argument. The exact meaning of ``equivalent'' depends on the type of object and is up to the programmer who defines a method for make-load-form. This allows the user to program the notion of ``similar as a constant'' (see section 25.1). The second value, called the initialization form, is a form that, when evaluated at load time, should perform further initialization of the object. The value returned by the initialization form is ignored. If the make-load-form method returns only one value, the initialization form is nil, which has no effect. If the object used as the argument to make-load-form appears as a constant in the initialization form, at load time it will be replaced by the equivalent object constructed by the creation form; this is how the further initialization gains access to the object. Two values are returned so that circular structures may be handled. The order of evaluation rules discussed below for creation and initialization forms eliminates the possibility of partially initialized objects in the absence of circular structures and reduces the possibility to a minimum in the presence of circular structures. This allows nodes in non-circular structures to be built out of fully initialized subparts. Both the creation form and the initialization form can contain references to objects of user-defined types (defined precisely below). However, there must not be any circular dependencies in creation forms. An example of a circular dependency: the creation form for the object X contains a reference to the object Y, and the creation form for the object Y contains a reference to the object X. A simpler example: the creation form for the object X contains a reference to X itself. Initialization forms are not subject to any restriction against circular dependencies, which is the entire reason for having initialization forms. See the example of circular data structures below. The creation form for an object is always evaluated before the initialization form for that object. When either the creation form or the initialization form refers to other objects of user-defined types that have not been referenced earlier in the compile-file, the compiler collects all of the creation and initialization forms. Each initialization form is evaluated as soon as possible after its creation form, as determined by data flow. If the initialization form for an object does not refer to any other objects of user-defined types that have not been referenced earlier in the compile-file, the initialization form is evaluated immediately after the creation form. If a creation or initialization form F references other objects of user-defined types that have not been referenced earlier in the compile-file, the creation forms for those other objects are evaluated before F and the initialization forms for those other objects are also evaluated before F whenever they do not depend on the object created or initialized by F. Where the above rules do not uniquely determine an order of evaluation, it is unspecified which of the possible orders of evaluation is chosen. While these creation and initialization forms are being evaluated, the objects are possibly in an uninitialized state, analogous to the state of an object between the time it has been created by allocate-instance and it has been processed fully by initialize-instance. Programmers writing methods for make-load-form must take care in manipulating objects not to depend on slots that have not yet been initialized. It is unspecified whether load calls eval on the forms or does some other operation that has an equivalent effect. For example, the forms might be translated into different but equivalent forms and then evaluated; they might be compiled and the resulting functions called by load (after they themselves have been loaded); or they might be interpreted by a special-purpose interpreter different from eval. All that is required is that the effect be equivalent to evaluating the forms. It is valid for user programs to call make-load-form in circumstances other than compilation, providing the argument's metaclass is not built-in-class or a subclass of built-in-class. Applying make-load-form to an object whose metaclass is standard-class or structure-class for which no user-defined method is applicable signals an error. It is valid to implement this either by defining default methods for the classes standard-object and structure-object that signal an error or by having no applicable method for those classes. See load-time-eval. In the following example, an equivalent instance of my-class is reconstructed by using the values of two of its slots. The value of the third slot is derived from those two values. (defclass my-class () ((a :initarg :a :reader my-a) (b :initarg :b :reader my-b) (c :accessor my-c))) (defmethod shared-initialize ((self my-class) slots &rest inits) (declare (ignore slots inits)) (unless (slot-boundp self 'c) (setf (my-c self) (some-computation (my-a self) (my-b self))))) (defmethod make-load-form ((self my-class)) `(make-instance ',(class-name (class-of self)) :a ',(my-a self) :b ',(my-b self))) This code will fail if either of the first two slots of some instance of my-class contains the instance itself. Another way to write the last form in the preceding example is (defmethod make-load-form ((self my-class)) (make-load-form-saving-slots self '(a b))) This has the advantages of conciseness and handling circularities correctly. In the next example, instances of class my-frob are ``interned'' in some way. An equivalent instance is reconstructed by using the value of the name slot as a key for searching for existing objects. In this case the programmer has chosen to create a new object if no existing object is found; an alternative possibility would be to signal an error in that case. (defclass my-frob () ((name :initarg :name :reader my-name))) (defmethod make-load-form ((self my-frob)) `(find-my-frob ',(my-name self) :if-does-not-exist :create)) In the following example, the data structure to be dumped is circular, because each node of a tree has a list of its children and each child has a reference back to its parent. (defclass tree-with-parent () ((parent :accessor tree-parent) (children :initarg :children))) (defmethod make-load-form ((x tree-with-parent)) (values `(make-instance ',(class-of x) :children ',(slot-value x 'children)) `(setf (tree-parent ',x) ',(slot-value x 'parent)))) Suppose make-load-form is called on one object in such a structure. The creation form creates an equivalent object and fills in the children slot, which forces creation of equivalent objects for all of its children, grandchildren, etc. At this point none of the parent slots have been filled in. The initialization form fills in the parent slot, which forces creation of an equivalent object for the parent if it was not already created. Thus the entire tree is recreated at load time. At compile time, make-load-form is called once for each object in the tree. All the creation forms are evaluated, in unspecified order, and then all the initialization forms are evaluated, also in unspecified order. In this final example, the data structure to be dumped has no special properties and an equivalent structure can be reconstructed simply by reconstructing the slots' contents. (defstruct my-struct a b c) (defmethod make-load-form ((s my-struct)) (make-load-form-saving-slots s)) This is easy to code using make-load-form-saving-slots. [Function] make-load-form-saving-slots object &optional slots This returns two values suitable for return from a make-load-form method. The first argument is the object. The optional second argument is a list of the names of slots to preserve; it defaults to all of the local slots. make-load-form-saving-slots returns forms that construct an equivalent object using make-instance and setf of slot-value for slots with values, or slot-makunbound for slots without values, or other functions of equivalent effect. Because make-load-form-saving-slots returns two values, it can deal with circular structures; it works for any object of metaclass standard-class or structure-class. Whether the result is useful depends on whether the object's type and slot contents fully capture an application's idea of the object's state. [change_end] ------------------------------------------------------------------------------- 23.5. Accessing Directories The following function is a very simple portable primitive for examining a directory. Most file systems can support much more powerful directory-searching primitives, but no two are alike. It is expected that most implementations of Common Lisp will extend the directory function or provide more powerful primitives. [Function] directory pathname &key A list of pathnames is returned, one for each file in the file system that matches the given pathname. (The pathname argument may be a pathname, a string, or a stream associated with a file.) For a file that matches, the truename appears in the result list. If no file matches the pathname, it is not an error; directory simply returns nil, the list of no results. Keywords such as :wild and :newest may be used in pathname to indicate the search space. [change_begin] X3J13 voted in March 1988 (PATHNAME-STREAM) to specify exactly which streams may be used as pathnames. See section 23.1.6. X3J13 voted in January 1989 (CLOSED-STREAM-OPERATIONS) to specify that directory is unaffected by whether the first argument, if a stream, is open or closed. If the first argument is a stream, directory behaves as if the function pathname were applied to the stream and the resulting pathname used instead. However, X3J13 commented that the treatment of open streams may differ considerably from one implementation to another; for example, in some operating systems open files are written under a temporary or invisible name and later renamed when closed. In general, programmers writing code intended to be portable should be careful when using directory. X3J13 voted in June 1989 (PATHNAME-LOGICAL) to require directory to accept logical pathnames (see section 23.1.5). However, the result returned by directory never contains a logical pathname. [change_end] ------------------------------------------------------------------------------- Implementation note: It is anticipated that an implementation may need to provide additional parameters to control the directory search. Therefore directory is specified to take additional keyword arguments so that implementations may experiment with extensions, even though no particular keywords are specified here. As a simple example of such an extension, for a file system that supports the notion of cross-directory file links, a keyword argument :links might, if non-nil, specify that such links be included in the result list. ------------------------------------------------------------------------------- -------------------------------------------------------------------------------